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Abstract. As people both reason and make decisions with uncertain geo- 
spatial data every day, it is important to understand the complexity of un- 
certainty, how it propagates through each dataset, and how to best visualize 
uncertainty to support reasoning and decision- making. When decisions are 
made from visualized geospatial data without the uncertainty explicitly 
mentioned or depicted with the dataset, it can lead to an inaccurate or mis- 
leading understanding of spatial patterns and processes. While this re- 
search area has expanded over the past two decades, much of the attention 
in evaluating uncertainty visualizations has been focused on identifying and 
measuring lower level perceptual processes. In contrast, we argue for a shift 
in attention towards assessing higher-level cognitive processes by keeping 
the focus on the user from the onset of visualization design to the final un- 
certainty visualization output. This article outlines two emerging research 
topics that will allow researchers to evaluate and promote these higher- level 
cognitive processes: identifying appropriate levels of precision when 
visualizing uncertainty and supporting a deeper comprehension of 
uncertainty. Currently, there is a lack of comprehensive empirical work that 
attempts to cognitively assess uncertainty visualization and decision- 
making through a human factors standpoint. A behavioral research ap- 
proach addressing uncertainty classification and deeper uncertainty can 
provide a necessary and significant contribution for more intuitive repre- 
sentations in current uncertainty visualization and research. 
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1. Introduction 



Utilization of geospatial information within analysis and decision- making 
without attention to uncertainty makes the assumption that the data in 
hand is accurate and reliable (Beard et al., 1991). However, uncertainty is 
inherent in all geospatial data (Duckham et al., 2001), including outputs 
derived from a geographic information system (Hope & Hunter, 2007). As 
peopleboth reason and make decisions with uncertain geospatial data every 
day, it is important to understand the complexity of uncertainty, how it 
propagates through each dataset, and how to best visualize uncertainty to 
support reasoning and decision- making. When decisions are made from 
visualized geospatial data without the uncertainty explicitly mentioned or 
depicted with the dataset, it can lead to an inaccurate or misleading under- 
standing of spatial patterns and processes. Thus, recent efforts have at- 
tempted to support the decision- maker through integration of uncertainty 
in data visualization. Hunter and Goodchild (1993) state that lack of proper 
attention to uncertainty can lead to the "use of wrong data, in the wrong 
way, to arrive at the wrong decision" (p. 55). It is essential that researchers 
address this complex topic that propagates within all geospatial data and 
research in order to develop better solutions and visualizations to support 
the user. 

Many uncertainty visualization techniques draw upon existing cartographic 
methods. However, due to a small number of empirical studies, little is 
known about the effectiveness of these techniques for visualizing data un- 
certainty. Additionally, whatever progress has been made on the empirical 
front has been split across several disciplines (MacEachren et al., 2005), 
and researchers have not been effective in systematically assessing and de- 
veloping formal ontologies that put individual evaluations into perspective. 
This lack of empirical testing may partially be due to the numerous and 
conflicting definitions of uncertainty (Aerts, Clarke, and Keuper, 2003; 
Pang, Wittenbrink and Lodha, 1997). For instance, Deitrick and Edsall 
(2008) f i nd that the term uncertai nty is an issue across multi pi e di sci pi i nes, 
where it is often defined using numerous terms: data quality, accuracy, pre- 
cision, error, vagueness, ambiguity, etc. For many authors, uncertainty is 
simply a configuration of many different views and definitions (Griethe & 
Schumann, 2005; Thomson et al., 2005) such as imprecision, error, subjec- 
tivity, and non-specificity. These different definitions area result of the lack 
of a formal taxonomy or ontology that would synthesize the topic across 
various domains and contexts. 

While this research area has expanded over the past two decades, much of 
the attention in evaluating uncertainty visualizations has been focused 
(perhaps unintentionally) on identifying and measuring lower- level percep- 



tual processes. In contrast, we argue for a shift in attention towards as- 
sessing higher-level cognitive processes by keeping the focus on the user 
from the onset of visualization design to the final uncertainty visualization 
output. This article outlines two research topics that will allow researchers 
to evaluate and promote these higher-level cognitive processes: supporting 
a deeper comprehension of uncertainty and identifying appropriate levels of 
precision when visualizing uncertainty. This article is not meant to provide 
a comprehensive overview but rather serves to identify two emerging areas 
of uncertainty visualization. 



2. Level of Precision 

Traditionally, classification of geospatial data uncertainty is often chosen 
and visualized by the map maker based upon the characteristics of the data 
without lending proper attention to the natural classes users may have for 
uncertainty or their requirements for making a decision. Depending upon 
the context, variations in the level of precision (i.e. number of classes and 
their respective ranges) may be necessary to better support the user in a 
more informed decision-making process. Potential research questions in 
this area may ask: Are there natural levels of precision for conceptualizing 
about geospatial data uncertainty? What levels of uncertainty are important 
for the user i n maki ng deci si ons? At what poi nt does i ncreasi ng the number 
of categories not make a difference in how they make a decision? Do we 
alter user behavior if we change the contextual scenario? To assess these 
questions, researchers need to incorporate behavioral studies with the end 
user in mind. 

Some users may prefer to reason about and make decisions with an un- 
classed color scheme (e.g. light to dark red representing lower to higher 
levels of uncertainty) that assigns a specific variation of the color to a single 
val ue of uncertai nty. There is a tradeoff perceptual ly, however, as the poi nts 
in the color spectrum can become impossible to discern. For example, as 
noted in Olson (1981), the use of more than nine categories in a bivariate 
choropleth color scheme quickly becomes confusing for the user. On the 
other hand, aggregating a range of uncertainty values into classes may not 
offer enough precision or the right distinctions the user needs to make a 
more informed decision. Ultimately, finding the right tradeoff will depend 
upon the requirements of the user. In Figure X Aerts, Clarke, and Keuper 
(2003) visualize uncertainty of urban growth using two different classifica- 
tion schemes (i.e. one contains two classes and the other five). I ncorporat- 
ing the end user may allow cartographers to choose the level of precision 



and classes in a way that is beneficial for the end user to reason and/ or 
make decisions from the visual result. 
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Figure L Images from Aerts, Clarke, and Keuper (2003) revealing two different 
classification schemes. 

As a si mpl e exampl e, someone may be i nterested i n the uncertai nty associ a- 
ted with the chance of rain in order to decide whether to bring an umbrella 
the foil owing day. Perhaps this person is only interested in two classes whe- 
re 70% is the critical value separating both classes. In another scenario, 
imagine a wildfire threatening your property. What classes would you likely 
want in order to make a decision of whether you evacuate the property? 
These consi derati ons suggest that users may appreci ate uncertai nty cl assi f i - 
cations that are flexible and interactive; in support of this notion, Kand- 
likar, Risbey, and Dessai (2005) note that "simple schemes that attempt to 
represent uncertainty in a uniform manner across many different 
contexts... may lead to biases... depending on how much detail is presented 
in the information" (p. 446). While several researchers have considered 
how the choice of classification system and color scheme can affect judg- 
ments and decisions made using maps (see, e.g., Brewer and Pickle 2002), 
more research is needed to extend these results to visualizations of 
uncertainty. 

Uncertainty may also be implicit in maps that do not explicitly represent 
uncertainty information. For example, map users' understanding of the 
physical processes associated with a hazard might shape their under- 
standing of how the certainty of this hazard varies across a map, even if the 
map does not explicitly show uncertainty information. Flood maps may be 
one case where uncertainty is implicit. As many people understand that 
locations closer to the water and have lower elevations are more likely to 



flood, they may intuitively assign higher certainty to flooding predictions 
for these areas than to adjacent areas that are higher or further from the 
water, even if the map shows both areas withi n the same hazard zone. Thus, 
uncertainty may already be implicit in many maps that do not explicitly 
include uncertainty representations. This implicit uncertainty may need to 
be quantified before the effect of explicitly visualizing uncertainty can be 
considered. 

Classification of uncertainty for visualization may prove particularly chal- 
lenging when the uncertainties that are being visualized are poorly defined, 
that is, when the probabilities or confidence intervals are themselves 
uncertain, or are completely unknown. The next section explores some of 
the chal I enges of visual i zi ng these deeper uncertai nti es. 



3. A Call For Visualizing Deeper Uncertainty 

A large number of uncertainty visualization studies employ evaluations in 
order to assess the effectiveness of the visualization techniques. We argue, 
however, that a large majority only assess whether participants can attain a 
surface understanding of uncertainty. In this approach, visualization tech- 
niques are evaluated based upon a participant's map reading skills, where 
the user attempts to identify the level of uncertainty on a map by visually 
matching the map graphics with the corresponding values in the legend. 
These types of assessments do not evaluate whether users obtain useful 
knowledge or utilize higher-level cognitive processes important in decision- 
making scenarios. When faced with important decisions, it is still under 
determination as to whether users actually grasp the complexity of 
uncertainty, or the deeper implication and intricacies of the data. Future 
research should attempt to support a deeper comprehension of data 
uncertainty by transitioning beyond current surface visualization techni- 
ques and evaluation methodologies. This section outlines a few case studies 
that evaluate surface level tasks and presents more complex evaluations 
and graphics in other research areas that have helped users grasp these 
deeper levels of uncertainty. 

In a study on visualizing uncertainty of urban growth, Aerts, Clarke, and 
Keuper (2003) recruited planners and decision- makers and asked them to 
make decisions based upon approximations and preferences related to the 
chance of urban growth. Uncertainty is displayed through color lightness 
for two separate display methods: static side-by-side comparison and ani- 
mation. I n this study, levels of uncertainty were aggregated and placed into 
set classes. On the simple approximations, participants only need to employ 
basic low-level perceptual processes to match colors between the map and 



legend to identify the corresponding uncertainty value. While users appear 
capable in associating particular numerical values with a specific level of 
uncertainty, it does not imply that they actually grasp the deeper meanings 
and intricacies of the data itself. Similarly, Hope and Hunter (2007) created 
a thematic map of land suitability for airport sites. Each areal region was 
shaded based on its suitability and an overlaid glyph bar indicated the 
amount of uncertainty. Users were tasked with deciding the optimal region 
for a new airport based on the combination of suitability and certainty, 
comparing two different areas. While this task forced participants to com- 
bine two types of data to make a decision, we argue, however, users still 
only need to perform a fairly simple map-reading task by matching the 
uncertainty representation with the value in the legend. 

These studies highlight the need to move towards research that captures a 
more profound understanding of the complex interaction among diverse 
uncertainties, cartographic communication, and decision making - in 
short, towards the study of what has been called 'deep uncertainty'. I n dis- 
cussions of deep uncertainty, "deep" can refer to profound indeterminacy in 
the data itself, to irreducible ignorance about the data, or - in the context of 
uncertainty communication - to the complexity and completeness with 
which an audience understands information about uncertainty that has 
been communicated to them. Most discussions of deep uncertainty focus on 
ignorance or indeterminacy, both of which define limits to what can be 
known. I n cases of indeterminacy, the fault is ontological (the true state of 
the world limits what can be known) whilein cases of ignorance, the fault is 
epistemological (the limits of human reasoning bound our knowledge). Ac- 
cording to Bankes (2002), the term "deep uncertainty" was first used by 
Nobel laureate and economist Kenneth Arrow in a talk on the Economics 
and I ntegrated Assessment of CI i mate Change that he gave at the Pew Cen- 
ter Workshop in 1999. While "deep uncertainty" is apparently of fairly re- 
cent coinage, the idea of multiple levels of uncertainty has a long history in 
economics, stretching back to at least Knight (1921), who distinguished 
uncertainty that is quantifiable using probability and statistics (which he 
termed "risk") from uncertainty that is not measurable (which he consi- 
dered uncertainty in the proper sense of the term). Perhaps following the 
lead of Dr. Arrow, much of the recent work on "deep uncertainty" has been 
in the field of climate change, with discussions considering ignorance and 
indeterminacy in both natural systems (Kandlikar et al. 2005) and human 
dimensions (Moser 2005). Researchers have also extended the discussion 
of deep uncertainty to other hazards and risks (see, e.g., Spiegelhalter and 
Riesh2011). 

Several researchers have explored the challenges inherent in the visual 
communication of uncertainty in general and deep uncertainty (ignorance 



and indeterminacy) in particular. For example, Spiegelhalter et al. (2011) 
suggest techniques for communicating probability using pie charts, bar 
charts, icon arrays and other techniques. In cases where probabilities are 
uncertain due to sampling error or other quantifiable limitations, they sug- 
gest using continuous fading or blurring to communicate this uncertainty. 
However, they also caution agai nst tryi ng to visualize truly deep uncertain- 
ty; noting that making a depiction of deep uncertainty more attractive may 
lead people to "believe it represents the whole truth rather than being a 
construction of limited knowledge and judgment," they contend that "the 
greatest chal I enge i s to make a vi sual i zati on that i s attracti ve and i nf ormati - 
ve, and yet conveys its own contingency and limitations" (Spiegelhalter et 
al. 2011 1400). Similarly, Kandlikar et al. (2005, 449-51) divide uncertainty 
about probabilities into six different levels, ranging from situations where a 
"robust, well-defended probability distribution" can be identified to "effec- 
tive ignorance"; intermediate levels include situations where minimum- 
maximum bounds, order of magnitude estimates, or the sign/ trend for the 
probability distribution can be identified with confidence. As understanding 
of the probability distribution becomes less precise, the authors recom- 
mend a corresponding reduction in the granularity or dimensionality of the 
representation (e.g., from a continuous probability density function, to a 
confidence interval, to a single order of magnitude estimate). While these 
examples do not explicitly consider the communication of uncertain spatial 
data on maps or other geographic representations, some of their general 
guidance on how to visually depict various levels of uncertainty can be 
transferred to a cartographic context. 

These attempts to formulate guidelines for communicating deep uncertain- 
ty do not explicitly consider how visualizations can be used to deepen un- 
derstanding of uncertainty. The distinction between surface and deep un- 
derstanding of uncertainty is closely associated with dual process theories 
of cognition, which suggest that humans process information using either 
heuristics and associations (quick and effortless but prone to errors and 
biases) or using a rule based, analytic approach (slow and effortful but less 
likely to commit logical errors). It also draws on levels of processi ng theory, 
which contends that deeper processing (which relates new information to 
prior knowledge) is more likely than shallower processing (which focuses 
on the surface or merely perceptual aspects of stimuli) to encourage enga- 
gement, understanding, and learning (Rapp 2005). While not explicitly 
addressing uncertainty, Rapp (2005) suggests that visualizations can en- 
courage deeper understanding of complex scientific information - parti cu- 
I arly when they are desi gned to be engagi ng and i nteracti ve. 

In a study by Mi call ef etal. (2012), the authors create visual representations 
to help participants grasp a deeper and more complex understanding of 



uncertainty and probability. Through crowdsourcing, the authors gather 
participants' understanding of six representations to assist in Bayesian 
reasoning that include area- proportional Euler diagrams, frequency grids, 
and combinations of both. Each of the six representations visualize five as- 
pects: 1) a woman with a positive mammography, 2) the population within 
this positive mammography that have breast cancer, 3) the population 
within this positive mammography that do not have breast cancer (false 
positive), 4) the population with a negative mammography who actually do 
have cancer (false negative), and 5) the rest of the population who correctly 
test negative for breast cancer. This multilevel representation allows users 
to better grasp the complexity of the context. More research is needed to 
transition useful visualizations I ike these into geospatial representations to 
assist users in gaining a deeper knowledge of uncertainty. 



4. Conclusion 

I n a weekly uncertainty visualization reading group, we have identified two 
largely missing areas in research (as outlined in this paper) that can benefit 
current approaches to assess and create intuitive and helpful uncertainty 
visualizations to promote reasoning and decision- making. There is a lack of 
comprehensive empirical work that attempts to cognitively assess uncer- 
tainty visualization and decision- making through a human factors stand- 
point. A behavioral research approach addressing uncertainty classification 
from the user's requirements and point of view as well as promoting a 
deeper comprehension of uncertainty can provide a necessary and signifi- 
cant contribution. When faced with important decisions, it is still under 
determination as to whether users actually grasp the complexity of uncer- 
tainty information, or the deeper implication and intricacies of the data. 
Many of the attempts to formulate guidelines for communicating a deeper 
uncertainty do not explicitly consider how visualizations can be used to 
deepen understanding of uncertainty"! t is important that techniques that 
aid probabilistic reasoning and decision making can benefit untrained peo- 
ple with different backgrounds and age ranges and, remain effective in situ- 
ations where little time and attention are available. " (Micallef et al., p. 
2542) 
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